Planning for Social Interaction in a Robot Bartender Domain

A robot coexisting with humans must not only be able to perform physical tasks, but must also be able to interact with humans in a socially appropriate manner. In many social settings, this involves the use of social signals like gaze, facial expression, and language. In this paper, we describe an application of planning to task-based social interaction using a robot that must interact with multiple human agents in a simple bartending domain. We show how social states are inferred from low-level sensors, using vision and speech as input modalities, and how we use the knowledge-level PKS planner to construct plans with task, dialogue, and social actions, as an alternative to current mainstream methods of interaction management. The resulting system has been evaluated in a real-world study with human subjects. Introduction and Motivation As robots become integrated into daily life, they must increasingly deal with situations in which socially appropriate interaction is vital. In such settings, it is not enough for a robot simply to achieve its task-based goals; instead, it must also be able to satisfy the social goals and obligations that arise through interactions with people in real-world settings. As a result, a robot not only requires the necessary physical skills to perform objective tasks in the world, but also the appropriate social skills to understand and respond to the intentions, desires, and affects of the people it interacts with. However, the problem of building a robot to meet the goals of social interaction presents several challenges for such a system. Not only does the robot require the ability to recognise and interpret appropriate multimodal social signals (e.g., gaze, facial expression, and language), but it must also generate realistic responses using similar modalities. Furthermore, while many interactions may lead to the same goal at the task level, the quality of those interactions may be greatly enhanced by getting the “people skills” right. To address this challenge, and focus our work, we are investigating the sub-problem of task-based social interaction in a bartending domain, by developing a robot bartender (Figure 1) that is capable of dealing with multiple human customers in a drink-ordering scenario. Interactions in this Copyright c © 2013, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Figure 1: The robot bartender and bar setting. scenario incorporate both task-based aspects (e.g., ordering and delivering drinks) and social aspects (e.g., managing multiple interactions). Moreover, the primary interaction modality in this setting is speech; humans communicate with the robot via speech and the robot must respond in a similar manner. This domain is also appealing since it is an excellent analogue for a range of common interaction and service scenarios in which an assistive robot must detect and respond to the needs of dynamically evolving groups of people in an unstructured, real-world environment. Since the ability to reason and plan is essential for a cognitive agent operating in a dynamic and incompletely known domain like the bartending scenario, high-level reasoning and action selection is a key component of our robot system. In particular, we use general-purpose planning techniques that can build goal-directed plans under many challenging conditions, especially in task-based contexts. Specifically, we use knowledge-level planning and the PKS planner (Petrick and Bacchus 2002; 2004), a choice that is motivated by PKS’s ability to work with incomplete information and sensing actions: not only will the robot have to perform physical tasks (e.g., handing a customer a drink), but it will often have to gather information it does not possess from its environment (e.g., asking a customer for a drink order). Moreover, since interactions will involve human customers, using speech as the main input modality, many of the planner’s actions will correspond to speech acts, which introduces a link to natural language processing. Indeed, re389 Proceedings of the Twenty-Third International Conference on Automated Planning and Scheduling